Practical Performance and Credit Assignment Efficiency of Analog Multi-layer Perceptron Perturbation Based Training Algorithms

Many algorithms have been recently reported for the training of analog multi-layer perceptron. Most of these algorithms were evaluated either from a computational or simulation view point. This paper applies several of these algorithms to the training of an analog multi-layer perceptron chip. The advantages and shortcomings of these algorithms in terms of training and generalisation performance and their capabilities in a limited precision environment are discussed. Extensive experiments demonstrate that a trade-off exists between the parallelisation of perturbations and the efficiency of credit assignment. Two semi-parallelisation heuristics are presented and are shown to provide advantages in terms of efficient exploration of the solution space and fewer credit assignment confusions.